
COMBINED COLOR SPACE MATRIX TRANSFORMATION 



AND FIR FILTER 



BACKGROUND OF THE INVENTION 

5 1 . Field of the Invention 

This invention relates to the field of video processing, and in particular to a video 
processor that includes a color space matrix transformation and finite impulse response (FIR) 
filter. 

10 2 . Description of Related Art 
FIR Filters: 

s = : Finite impulse response (FIR) filters are commonly used in video processing systems for 

; Q filtering picture element (pixel) values, particularly pixel values that are produced when an 
;Q image is scaled or otherwise transformed from its original form. For example, a decimation, or 

3 p 

ljS downsampling, of an image to produce a 4: 1 reduction in image size can be effected by merely 

Q choosing every fourth input pixel value to provide each output pixel value. Such a sampling, 

3 however, will generally produce visual anomalies and/or discontinuities, as the particular pixel 

;!? value selected may not be representative of the non-selected pixel values. An FIR filter provides 



f U for a weighted averaging of a plurality of pixel values to produce each pixel value, thereby 



upscaling application, wherein multiple output samples are produced for each input sample, the 
weighted averaging provides for an interpolated value for each output sample, wherein the 
inclusion of pixel values beyond the immediately adjacent input samples provides for a less- 
artificial-appearing filling of values between the input samples. 



provides an output pixel value that is a dependent upon six input pixel values, each pixel value 
being represented by three components, such as red-green-blue components in an RGB color 
space. In a conventional six-tap three-component FIR filter, the following transformation is 
effected: 




25 



A common FIR structure for video processing is a six-tap, three-component filter that 
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In the above equation, a^ corresponds to the X th component of the j m pixel value, c 00 -c 25 are the 
coefficients, or weights, for providing the weighted average, and e 0 -e 3 are the translation, or 
offset, terms applied to each output pixel component value a^-a^. Additional processing, 
including quantization, rounding, clamping, etc. are also subsequently applied, but are not 
illustrated herein, for ease of understanding. 

Note that, for effective filtering, an FIR filter of at least 3 taps, and preferably 6 or more 
taps are generally employed. At least eighteen multipliers and adders are required to effect the 
example 6-tap 3-component FIR filter. 



Color Space Transformation: 

Generally, different video processing applications employ different color space 
representations of the pixel values. For example, computer systems generally employ an RGB 
(red-green-blue) color space, wherein each pixel value is represented by a red component, a 
green component, and a blue component. A conventional broadcast television signal, on the 
other hand, encodes images using a luminance component (Y) and two chrominance components 
(U and V) corresponding to a YUV color space. A quadrature variation of YUV is the YIQ color 
space. The YCbCr color space is a scaled version of YUV that is used in digital video encodings. 
Other color space representations of an image are also common in the art. In VIDEO 
DEMYSTIFIED, by Keith Jack, published in 1966 by HighText Interactive, Inc. of San Diego, 
California, eight different color space standards are presented, with equations for converting 
from one color space to another. 

In traditional systems that support more than one color space, the transformation between 
color spaces is typically effected via hardware that is customized for the particular color space 
transformation. A customized design allows for a reduction in circuitry and the required area for 
this circuitry on an integrated circuit, but limits the applicability of the design to the particular 
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color space transformation. If multiple transformations are intended to be supported, a matrix 
transformation structure is generally employed: 
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where Ao-A 2 are the component values of the input pixel in the input color space, A'q-A^ are the 
component values of the output pixel in the output color space, D 0 -D 2 are the offsets associated 
with the input color components, B 0 -B 2 are the offsets associated with the output color 
components, and C 00 -C 22 are the coefficients of the particular transformation equation. For 
example, the conversion between conventional RGB and YCbCr color spaces is effected via the 
following equations. 

From an input pixel having R-G-B component values to an output pixel having Y-Cb-Cr 
component values: 
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From an input pixel having Y-Cb-Cr component values to an output pixel having R-G-B 
component values: 
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In a system that is configured to provide multiple color-space transformations, the 
different input-to-output color space transformations are performed by loading the appropriate 
coefficients and offsets to a 3x3 matrix multiplier and adder. Note that such an embodiment 
requires at least nine multipliers and adders, in addition to a preprocessor that subtracts the 
appropriate offsets (D 0 , D„ D 2 ) from the input components (e.g. Y-16, Cb-128, and Cr-128) of 
particular color spaces. 
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BRIEF SUMMARY OF THE INVENTION 
It is an object of this invention to reduce the circuitry and area required to effect color 
space transformation. It is a further object of this invention to optimize the use of the circuitry 
provided for FIR filtering. 
5 These objects and others are achieved by using the same circuitry for effecting FIR 

filtering and color space transformation. The input to a multiply-add array of a conventional FIR 
filter is appropriately multiplexed to provide the alternative uses of the FIR filter components. In 
this manner, for example, the same multiply-add array of components can be used for scaling an 
image and for transforming the image from one color space to another. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 
The invention is explained in further detail, and by way of example, with reference to the 
^ accompanying drawings wherein: 

ifl FIG. 1 illustrates an example block diagram of a video processing system in accordance with this 

ail 

lf\ invention. 

]|f FIG. 2 illustrates an example block diagram of a multi-use FIR filter combination in accordance 
111 with this invention. 

!□ Throughout the drawings, the same reference numerals indicate similar or corresponding 

? f ; features or functions . 

20 

DETAILED DESCRIPTION OF THE INVENTION 
FIG. 1 illustrates an example block diagram of an image processing system 100 in 
accordance with this invention. The processing system 100 is presented as an example 
application of this invention, although one of ordinary skill in the art will recognize that the 

25 invention is not limited to this example application. The processing system 100 allows multiple 
input image sources, of different formats, and in particular, images having pixel values using 
different color-space encodings. A source selector 1 10 selects a particular input image source, 
and provides the select image to a color-space converter 120. In this example, to minimize 
design complexity, the color-space converter 120 is configured to convert, as necessary, pixel 

30 encodings in one color space to an encoding in a particular color space that the remainder of the 
system is configured to process. That is, for example, in a computer-based display system, a 
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number of utility processes and application programs exist that are based on an RGB color space 
encoding. For example, image enhancement schemes that are configured to sharpen images, or 
remove noise from images, are commonly available for processing RGB encoded images. By 
converting other color space encodings into an RGB encoding, these existing utility processes 
5 and application programs can be employed. In a conventional image processing system, the color 
space conversion function is provided by special purpose hardware or by a 3x3 matrix 
multiplication, as discussed in the background to this invention. In accordance with this 
invention, the color-space converter 120 is configured to employ the multiplication and addition 
components of an FIR filter 140b, discussed further below. 

10 As is common in the art, in the example image processing system 100, a memory 130 is 
used to communicate the image data among processing blocks. This memory 1 30 may include, 
for example, line buffers for individual lines in the image, frame memory for sequential images, 

^ cache memory, and so on. In this example, the color space converter 120 is configured to 
*S provide the source image data to the FIR filter 140b, and the FIR filter 140b is configured to 
If? place the color-space-converted data in the memory 130. 

j~ a A scaler 150 is configured to provide optional scaling of the image from the memory 

IH 130. In accordance with this invention, the example scaler 150 is configured to use the same 
(3 multiply-add array of an FIR filter 140b as the color-space converter 120, via a multiplexer 140a, 

11 \ as discussed further below. In this example, the scaled version of the image, from the FIR filter 

i y 

2D 140b, is also stored in the memory 130, for subsequent processing by a display driver 160, for 
presentation of the image on a display 170. Note that other processes, such as quantization, 
rounding, clamping, sharpening, and so on may be applied to the scaled or unsealed data that is 
in the memory 130, as is common in the art, but are not illustrated in FIG. 1, for ease of 
understanding. 

25 A multiplexer 140a selects whether the input to the FIR filter 140b is from the color- 

space converter 120, or the scaler 150. Depending upon the particular configuration and 
supported functions of the processing system 100, the multiplexer 140a may introduce 
constraints on the use of the FIR filter 140b. For example, for maximum throughput of the 
system 100, the multiplexer 140a may be configured such that only one function, color-space- 

30 conversion or image-scaling, may be applied for a given input stream. That is, for example, in an 
RGB-based system, scaling may be provided only for images that are provided from an RGB 
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source, which does not require color-space conversion. If throughput is not critical, the system 
100 may be configured to set the multiplexer 140a to convert each input image to the select 
color-space, and, after the image is converted, to set the multiplexer 140a to scale the color- 
space-converted image. Other multiplexing schemes will be evident to one of ordinary skill in 
the art in view of this disclosure. For example, the system may be configured to control the 
multiplexer 140a so that data from the converter 120 and scaler 150 are alternately processed, 
thereby allowing continuous processing, albeit at a slower rate than a system that limits the 
processing by the FIR filter 140b to one input or the other for the entire process. 



FIG. 2 illustrates an example block diagram of a multi-use FIR filter combination 140 in 
accordance with this invention. The FIR filter combination 140 comprises the aforementioned 
FIR filter 140b, and a plurality of multiplexers 220, 230, 240, that are represented in FIG. 1 as 
the multiplexer 140a. 

In accordance with this invention, the multiplexers 220, 230, 240 are configured to 
provide an appropriate input to the FIR filter 140b to effect the intended transformation. In the 
example of FIG. 2, the input InB corresponds to the conventional input to an FIR filter to effect a 
scaling function, as discussed above with regard to the 6-tap, 3 -component FIR filter that 
provides the following matrix operation: 
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As illustrated in the example of FIG. 2, a series of delay elements 210 in the FIR filter 
140b provide the a 0 , a" 1 , . . . a' 5 pixel values, corresponding, for example, to a series of pixel 
values comprising a six-pixel length of a horizontal line. The coefficients c 00 -c 25 and offsets e 0 -e 3 
are set by the scaler 150 (of FIG. 1), based on the scaling factor and phase of the input and 
output pixel values, using algorithms that are common in the art. The above matrix 
multiplication and addition is effected via an array of multipliers and adders 250, as is common 
in the art for conventional FIR filters. 
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To provide color-space conversion of an input InA to the multiplexed FIR filter system 
140, the conventional 3x3 matrix operation is provided by the example 6-tap, 3-component FIR 
filter as follows. 

As presented above, the conventional 3x3 color-space conversion is provided by: 
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This conventional 3x3 color-space conversion can be shown to be equivalent to: 



By 



That is, the 3x3 color-space conversion coefficient matrix C 00 -C 22 is duplicated to fill the 
6x3 FIR filter coefficient matrix, and the input values \-A 2 and offsets D 0 -D 2 are provided as 
discrete entries. The YCbCr to RGB color-space conversion as presented in the background of 
the invention, for example, is recast to: 



By recasting the conventional 3x3 color-space conversion matrix operation to a 6x3 FIR 
filter matrix, the multiplexers 220, 230, 240 can be configured to provide the appropriate inputs 
to the example 6x3 FIR filter 140b of FIG. 2. As illustrated, when the mode signal is asserted to 
an opposite state to the state used to effect the conventional FIR filter function, the inputs to the 
multiply-add array 250 of the FIR filter 140b are switched by the multiplexer to provide the 
appropriate input values Ao-A 2 , output-offset values B 0 -B 2 duplicated coefficient values C 00 -C 22 , 
and input-offsets D 0 -D 2 to the 6x3 multipliers and adders to effect the 6x3 color-transformation 
matrix operation presented above. Note that, although the 3x3 color space transformation is 
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effected by a 3x6 multiply-add array 250, which is substantially larger than the conventional 3x3 
multiply-add array that is typically used to effect a 3x3 matrix operation, a savings in circuit-area 
is achieved, because the multiplexers 220, 230, 240 can be expected to consume less area than a 
conventional 3x3 multiply-add array. 

As would be evident to one of ordinary skill in the art, if a different sized FIR filter 140b 
is provided, the 3x3 color-space conversion matrix operation is recast as required to conform to 
the different sized FIR filter. A 4x4 FIR filter can be used, for example, by filling the parameters 
of the 3x3 color-space matrix directly into the 3x3 subset of the 4x4 FIR filter, and zero-filling 
the unused inputs. 

The foregoing merely illustrates the principles of the invention. It will thus be 
appreciated that those skilled in the art will be able to devise various arrangements which, 
although not explicitly described or shown herein, embody the principles of the invention and 
are thus within the spirit and scope of the following claims. 
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